Proactive cooling control using power consumption trend analysis

ABSTRACT

A fluid-cooled computer system includes a plurality of heat-generating components and a cooling system configured for supplying a cooling fluid at a controlled cooling fluid flow rate to cool the heat-generating components. A temperature-based cooling control circuit includes a temperature sensor configured for sensing a temperature of the heat-generating components and control logic for increasing a cooling fluid flow rate in response to the temperature exceeding a temperature threshold. A power-based cooling control circuit is configured for identifying and quantifying an increasing power consumption trend over a target time interval and, during a period that the temperature of the electronic device does not exceed the temperature threshold, increasing a cooling fluid flow rate to the electronic device in response to the magnitude of the increasing power consumption trend exceeding a power threshold. In one option, the fluid-cooled computer system is a server and the heat-generating components include a processor.

BACKGROUND

1. Field of the Invention

The present invention relates generally to power management and coolingin computer equipment.

2. Background of the Related Art

A data center is a facility where computer equipment and relatedinfrastructure are consolidated for centralized operation andmanagement. Computer equipment may be interconnected in a datacenter toproduce large, powerful computer systems that are capable of meeting thecomputing requirements of entities that store and process large amountsof data, such as corporations, web hosting services, and Internet searchengines. A data center may house any number of racks, with each rackcapable of holding numerous modules of computer equipment. The computerequipment typically includes a large number of rack-mounted serversalong with supporting equipment, such as switches, power supplies,network communications interfaces, environmental controls, and securitydevices. These devices are typically mounted in racks in a compact,high-density configuration to make efficient use of space whileproviding physical access and enabling the circulation of cool air.

Two important aspects of operating a datacenter are the management ofpower consumed by the equipment and the provision of adequate cooling.The large amount of rack-mounted computer equipment in a datacenter maycollectively consume a large quantity of power and generate a largeamount of heat. The infrastructure provided in a datacenter is intendedto support these significant power and cooling demands. For example, thedatacenter may provide electrical utilities with the capacity to power alarge volume of rack-mounted computer equipment, and a cooling systemcapable of removing the large quantity of heat generated by therack-mounted computer equipment. The cooling system in manyinstallations will also include a particular arrangement of equipmentracks into alternating hot aisles and cold aisles, and a computer roomair conditioner (“CRAC”) capable of supplying chilled air to the coldaisles. Meanwhile, chassis-mounted blower modules help remove heat fromthe racks and exhaust the heated air into the hot aisles.

The servers commonly used in datacenters are becoming more challengingto cool, as a result of parameters such as higher component packagingdensities, sharper variations in server workload, and a general demandto reduce energy consumption within cooling subsystems. Power managementin a server or among a group of servers addresses the balance betweenproviding greater amounts of cooling and driving up energy consumption.Providing more cooling than necessary will consume more energy thannecessary. Conversely, too little cooling can cause a temperaturethreshold to be exceeded and invoke performance-reducing power reductionmeasures. Exceeding such a temperature threshold may cause the server toperform at a lower performance level in order to reduce the heatgenerated and avoid component damage or data integrity issues. In orderto avoid exceeding such a temperature threshold even when a componentsuddenly goes from an idle state to maximum workload, systems typicallykeep critical components well below the allowable temperature thresholdsto provide an additional safety margin. Of course, maintainingcomponents at lower operating temperatures requires the consumption ofadditional power.

BRIEF SUMMARY

A system and method are disclosed for proactively cooling an electronicsystem in response to a dynamically computed power consumption trendover a moving time interval. In an example method, a temperature of anelectronic device is monitored. A cooling fluid flow rate to theelectronic device is increased in response to the temperature exceedinga temperature threshold. A power consumption of the electronic device isalso monitored. An increasing power consumption trend over a target timeinterval is identified and quantified. During a period that thetemperature of the electronic device does not exceed the temperaturethreshold, a cooling fluid flow rate to the electronic device isincreased in response to the magnitude of the increasing powerconsumption trend exceeding a power threshold.

An example of a fluid-cooled computer system includes a plurality ofheat-generating components and a cooling system configured for supplyinga cooling fluid at a controlled cooling fluid flow rate to cool theheat-generating components. The fluid-cooled computer system may be, forexample, a server, wherein the plurality of heat-generating componentsincludes a processor. A temperature-based cooling control circuitincludes a temperature sensor configured for sensing a temperature ofthe heat-generating components and control logic for increasing acooling fluid flow rate in response to the temperature exceeding atemperature threshold. A power-based cooling control circuit isconfigured for identifying and quantifying an increasing powerconsumption trend over a target time interval and, during a period thatthe temperature of the electronic device does not exceed the temperaturethreshold, increasing a cooling fluid flow rate to the electronic devicein response to the magnitude of the increasing power consumption trendexceeding a power threshold.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a schematic diagram of an electronic system having a coolingsystem that combines the use of temperature and power feedback incooling a unit of Information Technology Equipment (ITE).

FIG. 2 is a schematic diagram of a specific example embodiment whereinthe ITE of FIG. 1 is a blade server.

FIG. 3 is a graph plotting an instantaneous ITE power consumption and anITE power consumption trend computed as a function of instantaneous ITEpower consumption.

FIG. 4 is a graph provided as a visual aid in discussing how atemperature threshold may be raised when power consumption trendanalysis is used to proactively increase fan speed.

FIG. 5 is a flowchart of a method for controlling the cooling fluid flowrate in an electronic system on the basis of both temperature and powerconsumption trend analysis.

DETAILED DESCRIPTION

The present invention includes systems and methods for proactivelyincreasing a cooling fluid flow rate directed to a heat-generatingcomponent in response to an increasing power consumption trend of theheat-generating component. In an example system, a power-based coolingcontrol circuit and a temperature-based cooling control circuit sharecontrol of a cooling subsystem to control the cooling fluid flow rate.The power-based cooling control circuit monitors the real-time,instantaneous power consumption of one or more of its heat-generatingcomponents, identifies and quantifies an increasing power consumptiontrend over a moving target time interval of a finite but non-zeroduration, and selectively adjusts the cooling fluid flow rate inresponse to the computed power consumption trend exceeding a powerthreshold. In one embodiment, the target time interval may have a fixedlength within the range of between 5 and 30 seconds. The target timeinterval is of sufficient length to minimize the influence of powerspikes of shorter duration. As an option, power spikes of less than apredefined duration may even be ignored, so as to have no effect on thecooling fluid flow rate.

By quantifying the power consumption trend, the magnitude of the powerconsumption trend provides an indicator or prediction of imminentheating. One or more predefined power thresholds may be set to determinean appropriate cooling system response commensurate with an amount ofheating that may result from the increasing power consumption trend. Bymaking the cooling system responsive to the increasing power consumptiontrend, a cooling response is initiated before temperatures have risenappreciably as a result of the increasing power trend. The coolingresponse may therefore be more mild than waiting for a componenttemperature to reach a temperature threshold. Since fan powerconsumption can be an exponential function of fan speed, reducing themaximum fan speed can significantly reduce cooling costs. The sharedcontrol of the power-based cooling control circuit and temperature-basedcooling control circuit further reduces the likelihood of a criticaltemperature threshold from occurring. The proactive cooling may alsoallow a warning temperature threshold to be safely increased, thusdecreasing a margin between the critical temperature threshold and thelower, warning temperature threshold.

FIG. 1 is a schematic diagram of an electronic system 10 having acooling subsystem 40 that combines the use of temperature and powercontrol feedback to cool a unit of Information Technology Equipment(ITE) 12. The ITE 12 may be embodied, for example, as a computerhardware device, such as a rack-mounted server. The ITE 12 includes oneor more heat-generating components 14 that consume electrical power andgenerate heat in relation to the power consumed. The cooling subsystem40 cools the ITE 12 by directing a cooling fluid, such as air or aliquid coolant, to the ITE 12 at a controlled cooling fluid flow rate.For example, the cooling subsystem 40 may circulate air as the coolingfluid, using one or more fans to cool the ITE 12. The fans may includean on-board cooling fan provided on the ITE 12 or an external devicesuch as a chassis-mounted blower module to drive airflow through aplurality of ITE units. Alternatively, the cooling subsystem 40 maycirculate a chilled liquid coolant as the cooling fluid through a heatexchanger in direct mechanical and thermal contact with theheat-generating components 14. An example application combining the useof air and liquid coolant is a rear-door air-to-liquid heat exchanger ona rack of computer equipment, wherein airflow passing through the rackremoves heat by convection and the heated air is cooled by the rear-doorheat exchanger.

Whether using air, liquid coolant, or both as the cooling fluid, therate of cooling is related to the cooling fluid flow rate. For example,cooling increases as a result of increasing an airflow rate through theITE 12 or increasing a liquid flow rate within a liquid-cooled heatexchanger. The cooling subsystem 40 includes a flow controller 42 forenforcing a selected cooling fluid flow rate. For example, if thecooling fluid is air, the flow controller 42 may control the flow rateof the air through the ITE 12 by controlling a fan speed. Alternatively,if the cooling fluid is a liquid coolant, such as water, in aclosed-system heat exchanger, the flow controller 42 may control theflow rate of the liquid coolant using a variable pump. However, thecooling subsystem 40 has a finite limit on the rate at which it iscapable of flowing cooling fluid. In the case of a fan, the maximum flowrate and corresponding maximum cooling will occur when the fan(s) areoperating at their maximum fan speed. In the case of a liquid coolantheat exchanger, the maximum flow rate and corresponding maximum coolingwill occur when the liquid coolant is circulated at the maximum rateprovided by the cooling subsystem 40, which may be determined by theoperational limits of a liquid pump. The cost of operating the coolingsubsystem 40 generally increases with increasing flow rate. For example,the power consumed by a fan and the associated monetary cost ofoperating the fan is generally a cubic function of the fan speed, suchthat energy costs associated with cooling the ITE 12 may increaseexponentially with fan speed.

The system 10 of FIG. 1 includes two subsystems for selecting the flowrate to be enforced by the flow controller 42 of the cooling subsystem40. The two subsystems are embodied here as a temperature-based coolingcontrol circuit 20 and a power-based cooling control circuit 30, whichare independently in communication with the flow controller 42 of thecooling subsystem 40. As explained in further detail below, thetemperature-based cooling control circuit 20 and the power-based coolingcontrol circuit 30 share control of the cooling fluid flow rate by eachselectively adjusting the flow rate to be enforced by the flowcontroller 42. The temperature-based cooling control circuit 20 respondsto the temperature exceeding one or more predefined temperaturethresholds, such as a warning temperature threshold and a higher,critical temperature threshold. The power-based cooling control circuit30, however, responds proactively to an increasing power consumptiontrend before the temperature has a chance to reach a higher temperaturethreshold in response to the power consumption trend. By respondingproactively to increased power consumption, it is possible to prevent ordelay the temperature from exceeding the next higher temperaturethreshold. Cooling costs may also be reduced by reducing the maximumflow rate required to cool the heat-generating components 14 over agiven timeframe. For example, increasing the fan speed in response to anincreasing power consumption trend, before a next higher temperaturethreshold is reached, can reduce the maximum fan speed at which the fanis operated in a given timeframe, as compared with waiting until atemperature threshold has been reached to increase the fan speed.

The temperature-based cooling control circuit 20 includes a temperaturesensor 22, a threshold comparator circuit 24, and a temperature-basedcooling rate selector 26. The temperature sensor 22 may be in directthermal contact with one of the heat-generating components. Thetemperature sensor 22 may be positioned to sense the temperature of oneof the heat-generating component 14 expected to reach the highestoperating temperatures. For example, a processor typically runs hotterthan other components in a computer system, in which case a built-intemperature sensing diode or other integrated temperature sensorincluded on the processor may be used as the temperature sensor 22. Thethreshold comparator 24 and temperature-based cooling rate selector 26may each comprise control logic for performing their functions. Thethreshold comparator 24 is in communication with the temperature sensor22 for comparing the sensed temperature with one or more predefinedtemperature thresholds. The temperature-based cooling rate selector 26may then request a cooling rate on the basis of that comparison.

The temperature-based cooling rate selector 26 outputs a signal 43requesting a cooling rate (or adjustment to the cooling rate) to thecooling subsystem 40 to be enforced by the flow controller 42. Thesignal 43 may reflect an amplitude of the output from thetemperature-based cooling control circuit 20. The requested cooling ratemay be specified quantitatively, such as by a volumetric flow rate ofliquid coolant or a selected fan speed. The requested cooling rate mayalternatively be specified as one of a plurality of different predefinedlevels. For example, a fan included with the cooling subsystem 40 may beoperable at discrete fan speed levels, and the requested cooling ratemay specify a specific fan speed level. The requested cooling rate mayalternatively be a specified adjustment, such as by specifying “more” or“less” cooling, in response to which the flow controller 42 may increaseor decrease the fluid flow rate by one or more predefined steps.

In one implementation, a single temperature threshold is provided, sothat the temperature-based cooling control circuit 20 will only causethe cooling rate to increase if and when the temperature reaches thatsingle temperature threshold. The single temperature threshold may beselected as an upper temperature limit, such as a critical temperaturethreshold, and the response from the cooling subsystem 40 may be tomaximize the cooling fluid flow rate if and when the sensed temperaturereaches the upper temperature limit. If the maximum cooling fluid flowrate is insufficient to cool the ITE 12, then the ITE may be configuredto automatically enter a reduced power mode, or even shut down ifnecessary to avoid unsafe operating temperatures.

In another implementation, a plurality of different thresholds isprovided, and the cooling fluid flow rate is increased in response toeach successive threshold that is reached. For example, the flowcontroller 42 may be configured to operate at a first level in responseto exceeding a first temperature threshold, a second, higher level inresponse to exceeding a second temperature threshold greater than thefirst level, and so forth. Regardless of whether one or multipletemperature thresholds are provided, the cooling fluid flow rate isincreased only after the ITE 12 has reached a threshold value, which isafter the power consumption has previously increased to cause thetemperature of the ITE 12 to reach that temperature threshold value.

Whether the embodiments use a single temperature threshold, a pluralityof temperature thresholds, or continuous adjustments of fans speed as afunction of temperature, the power-based cooling control circuit mayproduce an output to the fan speed in response to detecting anincreasing power consumption trend. During a sudden increase in powerconsumption, the power-based cooling control circuit may produce asubstantial output to a fan speed controller. Still, if the powerconsumption reaches a new steady state value (i.e., the increasing powerconsumption trend stops or nears zero), then the power-based coolingcontrol circuit will contribute little or no signal to the fan speedcontroller and the fan speed will be substantially determined by theoutput of the temperature-based cooling control circuit.

The power-based cooling control circuit 30 also communicates with theflow controller 42 for selectively adjusting the flow rate. However, thepower-based cooling control circuit 30 proactively increases therequested flow rate in response to power consumption, before atemperature threshold is reached. By identifying and quantifying anincreasing power trend over a target time interval, the power-basedcooling control circuit 30 is able to anticipate the need for morecooling and to increase airflow accordingly. The power-based coolingcontrol circuit 30 may proactively increase cooling by increasing thecooling fluid flow rate before exceeding a temperature threshold, toavoid or at least delay exceeding the temperature threshold as comparedwith no increase in cooling. If the temperature-based cooling controlcircuit 20 is responsive to a plurality of different temperaturethresholds, the power-based cooling control circuit 30 may proactivelyincrease cooling when the temperature is between two successivetemperature thresholds, so as to delay exceeding each successivetemperature threshold and possibly to prevent the temperature fromexceeding a next temperature threshold.

In the ITE 12 of FIG. 1, the illustrated components of the power-basedcooling control circuit 30 are a power sensor 32, a power consumptiontrend analyzer 34, and a power-based cooling rate selector (output) 36.The power sensor 32 may be a physical sensor in electrical communicationwith one or more of the heat-generating components 14. As illustrated inFIG. 1, the power sensor 32 may sense a net power consumption of theentire group of heat-generating components 14 on the ITE 12. The powersensor 32 may be provided in-line with a power supply to a motherboard,for example. Alternatively, the power sensor 32 may be provided to sensethe power consumption of a specific one of the heat-generatingcomponents 14, such as a processor on a system board, wherepredominantly all of the heat generated by the ITE 12 is related to thepower consumption of that component. The power sensor 32 may sensecurrent flow as an indication of power, since power consumption isdirectly related to current flow. The power sensor 32 outputs the sensedpower consumption, which is a dynamic and constantly varying parameter,to the power consumption trend analyzer 34.

The power-based cooling control circuit 30 does not increase therequested flow rate directly in response to an instantaneous increase inpower consumption detected by the power sensor 32. Rather, the powerconsumption trend analyzer 34 is provided to analyze the powerconsumption over a time interval to identify and quantify a powerconsumption trend over that time interval. In one embodiment, the timeinterval may be between about 5 and 30 seconds. This is a moving timeinterval that may be frequently or continuously updated. For example,every second or every tenth of a second, the power consumption trendanalyzer 34 may compute an updated power consumption trend for the timeinterval leading up to that instant. Further discussion of the powerconsumption trend analysis is provided below with reference to FIGS. 3and 4.

On the basis of the power consumption trend, the P-based cooling rateselector 36 outputs a signal 41 requesting a cooling rate to be enforcedby the flow controller 42. The signal 41 may be representative of themagnitude of the power consumption trend. The flow controller 42 mayreceive and enforce the cooling rate requests from the power-basedcooling control circuit 30 in the same way that the flow controller 42receives and processes the cooling rate requests from thetemperature-based cooling control circuit 20. The requested cooling ratemay be specified quantitatively, as one of a plurality of differentpredefined levels or comparatively as by specifying “more” or “less”cooling.

Arbitration logic (not shown) may be included in case of any conflictbetween a cooling rate requested by the temperature-based coolingcontrol circuit 20 and a cooling rate requested by the power-basedcooling control circuit 30. For example, if the temperature-basedcooling control circuit 20 and power-based cooling control circuit 30request two different cooling rates, the flow controller 42 may handlethe conflicting request by deferring to the higher of the two requestedcooling rates, or by giving requests from one of the two circuits 20, 30priority over requests from the other of the two circuits 20, 30.Alternatively, the flow controller 42 may simply process and enforcerequested cooling rate selectors in the order in which they arereceived, without any arbitration, and without deference to whether therequested cooling rates come from the temperature-based cooling controlcircuit 20 or the power-based cooling control circuit 30. For example,as power consumption increases over time, the power-based coolingcontrol circuit 30 may request a cooling rate increase before thetemperature reaches a first temperature threshold.

If the temperature eventually reaches the first temperature threshold,the temperature-based cooling control circuit 20 may then request acooling rate increase to slow down the rate at which the ITE 12 heatsup. In one embodiment, the cooling rate may be solely determined by thepower-based cooling control circuit 30, unless and until a temperaturethreshold is reached. In another embodiment, the cooling rate may besolely determined by the temperature-based cooling control circuit 20,unless and until a power threshold is reached to which the power-basedcooling control circuit 30 is responsive. If the temperature thresholdis reached, such as a maximum or critical temperature, thetemperature-based cooling control circuit 20 may override any previouslyselected fluid flow rate. The temperature-based cooling control circuit20 in that case may take over control of the cooling subsystem 40 inorder to ensure unsafe operating levels are not exceeded.

FIG. 2 is a schematic diagram of a more specific example embodiment ofthe electronic system 10, wherein the ITE 12 is a blade server 12A. Theheat-generating components 14 in this example are components of amotherboard 50. These heat-generating components 14 include a centralprocessing unit (CPU) 52, system memory in the form of random accessmemory (RAM) 54, and one or more expansion cards 56. Among thesecomponents, the CPU 52 is typically going to operate at hottertemperatures than the other components. Thus, the temperature sensor 22is located directly on the CPU 52. The power sensor 32 may be physicallylocated anywhere within the blade server 12A, and may sense the powerconsumption of the entire motherboard 50. Alternatively, the powersensor 32 may be configured to sense the power consumption of the CPU 52only, since the CPU 52 may consume more power than the other componentsalso by a significant margin. A power supply 58 may include circuitry toconvert alternating current (AC) from an AC source (not shown) to directcurrent (DC) at a plurality of different DC voltages for supplying thevarious components of the motherboard 50 according to their differentpower requirements.

An integrated management controller 55 is provided on the motherboard50. The management controller 55 may be a Baseboard ManagementController (BMC) or an Integrated Management Module (IMM), for example.The management controller 55 manages various aspects of the server'soperation at the server level, and may communicate with a chassismanagement module in a chassis having a plurality of the blade servers12A. While the temperature sensor 22 and power sensor 32 are separatefrom the management controller 55, other elements of both thetemperature-based cooling control circuit 20 and power-based coolingcontrol circuit 30 of FIG. 1 may be included on the managementcontroller 55. For example, control logic for implementing the thresholdcomparator 24 and temperature-based cooling rate selector 26 and controllogic for implementing the power consumption trend analyzer 34 and theP-based cooling rate selector 36 (See FIG. 1) may all reside “on-chip”in this embodiment of the electronic system 10. Thus, the signal outputsof the temperature sensor 22 and power sensor 32 are fed to themanagement controller 55. The management controller 55 may combine theuse of the threshold comparator 24, temperature-based cooling rateselector 26, power consumption trend analyzer 34, and P-based coolingrate selector 36 from FIG. 1 to selectively output either a requestedT-dependent flow rate or P-based flow rate to the flow controller 42.The manner in which the requested T-dependent flow rate or P-based flowrate is as described above with reference to FIG. 1.

FIG. 3 is a graph 60 plotting an instantaneous ITE power consumption 61and an ITE power consumption trend 62 computed as a function of theinstantaneous ITE power consumption. The instantaneous ITE powerconsumption 61 and ITE power consumption trend 62 are plotted as afunction of time T in seconds(s). At any time T, the magnitude of theinstantaneous ITE power consumption 61 is indicated by the left-sidevertical axis 63 and the value of the ITE power consumption trend 62 isindicated by the right-side vertical axis 64. The instantaneous ITEpower consumption 61 increases from a quasi-steady state value of 158Watts (W) to 321 W at time T=10 s. The ITE power consumption trend 62relates to the rate of change of the instantaneous ITE power consumption61, and may be an amplitude of the signal 41 output by the power-basedcooling control circuit 30 to the flow controller 42 in FIG. 1.

In this embodiment, the value of the ITE power consumption trend 62 iscomputed as PT=(P2−P1)/P1, where P1 is the moving average powerconsumption over a target time interval and P2 is the moving averagepower consumption over a shorter time interval. Specifically, P1 is themoving average power consumption over a ten-second time interval and P2is the moving average power consumption over a three-second timeinterval in this example. Thus, a positive value of the ITE powerconsumption trend 62 indicates that the average power consumption overthe preceding three seconds is higher than the average power consumptionover the preceding ten seconds. This is just one example of a powerconsumption trend equation; any of a variety of different powerconsumption trend equations may be used, wherein the power consumptiontrend is computed on the basis of a non-zero time interval. Since thepower consumption trend occurs over a non-zero time interval, the powerconsumption trend provides a better indication of power consumption thatmay lead to imminent heating, as it is possible for the instantaneouspower consumption to vary dramatically without leading to a significanttemperature increase.

An example cooling response chart 66 is also included in FIG. 3. Thecooling response chart 66 indicates an example of a predefined fanresponse to the ITE power consumption trend 61. The flow rate iscontrolled by a fan speed. The fan speed may be adjusted in discretesteps (i.e., predefined fan speed states that may be described by a fanspeed table) each time the magnitude of the power consumption trendreaches one of a plurality of predefined power threshold values. In thisexample, the power consumption trend values are at 25%, 50%, 75%, and100%. If the value of the power consumption trend PT reaches 25% (i.e.0.25 on the right-side vertical axis 64) then the fan speed isincremented 1 step. If the value of the power consumption trend PTreaches 50%, then the fan speed is incremented to 2 steps relative tothe current or last known operating state. If the value of the powerconsumption trend PT reaches 75%, then the fan speed is incremented to 3steps. If the value of the power consumption trend PT reaches 100%, thenthe fan speed is incremented to 4 steps. Note that the example powerconsumption trend equation PT=(P2−P1)/P1 allows for PT values of greaterthan 100%. Also, it is possible for the power consumption to increasegradually enough for the value of the power consumption trend PT to staybelow the lowest threshold (25% in this example). In that case, thetemperature-based cooling control system will continue to control thecooling rate according to the predefined temperature thresholds.

In the example graph 60 of FIG. 3, the instantaneous power consumption61 jumps upward from 158 W to 321 W at time T=10 s. At the instant whenT=10 s, the average power consumption for the past ten seconds (P1) andthe average power consumption for the past three seconds (P2) are bothstill 158 W, since the ITE has been operating at a quasi steady statevalue of about 158 W for at least the preceding ten seconds. Thus, thevalue of the ITE power consumption trend 62 at T=10 s is still zero, butthe power consumption trend curve 62 increases sharply. At time T=11 s,the computed values of P1 (average power consumption over ten seconds)and P2 (average power consumption over three seconds) would be about 174W and 212 W, respectively. This gives the ITE power consumption trend 62a value of about 0.22=(212 W−174 W)/174 W, or 22%, which is still lessthan the minimum power consumption threshold of 25%. At time T=12 s, theITE power consumption trend 62 has increased to a value of about 0.40,or 40%, and the fan speed will have been increased automatically as theITE power consumption trend 62 increases above the 25% threshold betweenT=11 s and T=12 s. At time T=13 s, the ITE power consumption trend 62has increased to about 0.55 (55%), which is a local-maximum value of theITE power consumption trend 62. The fan speed will have been increasedautomatically to two fan speed steps as the ITE power consumption trend62 increases above the 50% threshold between T=12 s and T=13 s.

After time T=13 s, the power consumption trend curve 62 begins a steepdecline, reflecting a slowing of the rate at which the average powerconsumption is increasing over the moving ten-second time interval. Bytime T=20 s, the value of the power consumption trend curve is back tozero. Optionally, as the ITE power consumption trend 62 begins todecline at T=13 s and passes back through the thresholds in reverseorder, the methods of the present invention may also be utilized toreduce fan speed intelligently.

By responding to the ITE power consumption trend 62 rather than to justany increase in the instantaneous ITE power consumption 61, thepower-based cooling control circuit 30 (FIG. 1) more intelligentlymanages cooling. For example, by following the ITE power consumptiontrend 62, the associated cooling response may respond minimally, if atall, to very brief power increases (i.e. power spikes). Power spikes ofeven high power magnitudes may be so short as to not greatly affect heatgeneration in the ITE. According to the power consumption trend equationPT=(P2−P1)/P1, a brief power spike of even a large magnitude may notcause the value of PT to exceed 25%, thus eliciting no fan response inthis example. This de-sensitivity to very short power fluctuationsavoids the fan or liquid coolant pump from fluctuating rapidly orhaphazardly in response to every brief change in power. This helpsprevent unnecessarily straining the cooling equipment or unnecessarilyconsuming higher power in cooling the ITE.

FIG. 4 is a graph 70 provided as a visual aid in discussing how atemperature threshold may be desirably raised in a system that alsoincludes the power-based cooling control circuit to proactively increasefan speed. A temperature response 67 and temperature axis 68 areprovided on the same graph 70. The ITE power consumption trend 62 and PTaxis 64 are carried over from FIG. 3 for reference. The temperatureresponse 67 indicates an expected temperature increase (“delta T”) inresponse to the increased power at time T=10 s, but in the absence ofthe proactive power-responsive cooling described with respect to FIG. 1.This expected temperature response in the absence of proactive coolingwould lead to an expected 8.5 Celsius (C) rise in the first five seconds(i.e. from T=10 s to T=15 s), and a 12 C rise in twelve seconds. Usingonly a temperature-based cooling control such as provided by thetemperature-based cooling control circuit 20 of FIG. 1, no fan speedincrease would occur prior to a predefined temperature threshold,despite the increase in average power consumption reflected in the PTcurve 62.

Some computer systems incorporate a temperature threshold (e.g. “Twarn”in the graph 70) that is lower than a critical temperature (“Tcrit”),for triggering a significant system response when approaching an unsafetemperature. For example, exceeding a temperature threshold Twarn maycause the system to automatically invoke a reduced power state, andexceeding an even higher threshold (e.g. “Tcrit” in the graph 70) maycause the system to automatically invoke an emergency action, such as topower down the system. Another temperature threshold (e.g. “Tcontrol” inthe graph 70) that is even lower than Tcrit or Twarn may also be set, toinitiate a cooling response well in advance of exceeding Twarn or Tcrit.In the example chart 70 of FIG. 70, Tcontrol is set 12 degrees belowTwarn, which would require increasing the fan speed at about T=11 sunder the consumption curve of FIG. 3. That would cause the temperatureto “break” prematurely, as indicated at 67B in FIG. 4.

Providing a large margin between Twarn and Tcontrol can lead tohigher-than-necessary fan speeds and associated fan power consumption atlower utilization states below Twarn or Tcrit. By combining theproactive power-based cooling control to temperature-based coolingcontrol, the increase in temperature may be anticipated earlier by thesystem based on the power consumption trend 62, so that fan speed may beincreased sooner and more gradually. This proactive cooling therebyavoids the need for such a large safety margin between Tcontrol andTwarn, allowing the temperature threshold Tcontrol to be increased to avalue closer to Twarn. For example, as shown in FIG. 4, the usual12-degree margin between Twarn and Tcontrol is reduced to about 6degrees in this example when proactive power-based cooling is provided.The proactive power-based cooling results in a more gradual cooling thatstarts at lower temperatures, as indicated at 67C, so that the fan speedcan be increased sooner and more gradually, avoiding peak fan speeds.

FIG. 5 is a flowchart generally outlining an example method forcontrolling the cooling fluid flow rate in an electronic system on thebasis of both temperature and power consumption trend analysis. Detailsregarding specific steps of the flowchart may be aided by reference tothe discussion of FIGS. 1-4, above. In step 100, temperature of one ormore heat generating devices is monitored. In step 102, power of the oneor more heat-generating devices is monitored. Steps 100 and 102 may beconcurrently performed, such that dynamic temperature and powermeasurements are obtained. The temperature of the heat-generatingdevices is expected to be related to the power consumed by theheat-generating devices. A power consumption trend is computed in step104, over a moving target time interval. Computing the power consumptiontrend includes identifying and quantifying the power consumption trend.A signal may be output in relation to the magnitude of the quantifiedpower consumption trend.

In the outlined method, the temperature is used as a parameter indynamically selecting a cooling rate. The power consumption trendcomputed in step 104 is also used as a parameter in dynamicallyselecting a cooling rate. The instantaneous power consumption monitoredin step 102, however, is not directly used in selecting a cooling rate,since large spikes or other momentary fluctuations in the instantaneouspower consumption would typically not warrant an immediate change in thecooling rate. In particular, the temperature is compared to atemperature threshold in conditional step 106. If the temperature hasreached (i.e., is equal to or greater than) the temperature threshold,then an increased cooling fluid flow rate is requested in step 110.Otherwise, the temperature is below the temperature threshold, andconditional step 108 queries whether the magnitude of the powerconsumption trend has reached (i.e., is equal to or greater than) thepower consumption trend threshold. If the power consumption trend hasreached the power consumption trend threshold, then an increased coolingfluid flow rate is requested per step 110. The magnitude of therequested cooling increase in step 110 may be dependent on thetemperature if the increase is due to the temperature exceeding thetemperature threshold per step 106. The magnitude of the requestedcooling increase in step 110 may instead be dependent on the computedpower consumption trend if the increase is due to the power consumptiontrend exceeding the power consumption trend threshold per step 108.

As outlined in this flowchart, the value of the power consumption trendmay cause an increase in the cooling fluid flow rate per conditionalstep 108, during a period that the temperature has not exceeded thetemperature threshold per step 106. Thus, the power consumption trend isused proactively to initiate a cooling response before the temperaturethreshold is ever reached. This proactive, power-based control of thecooling system provides an earlier cooling system response than merelyrelying on temperature thresholds, since an increasing power consumptiontrend provides an early warning or prediction of an increasingtemperature. Meanwhile, the computation of the power consumption trendhelps to ensure that an early cooling system response is warranted, andhelps prevent cooling system fluctuations from momentary powerfluctuations in the instantaneous power consumption computed in step102.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,components and/or groups, but do not preclude the presence or additionof one or more other features, integers, steps, operations, elements,components, and/or groups thereof. The terms “preferably,” “preferred,”“prefer,” “optionally,” “may,” and similar terms are used to indicatethat an item, condition or step being referred to is an optional (notrequired) feature of the invention.

The corresponding structures, materials, acts, and equivalents of allmeans or steps plus function elements in the claims below are intendedto include any structure, material, or act for performing the functionin combination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but it is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

1-14. (canceled)
 15. A fluid-cooled computer system, comprising: aplurality of heat-generating components; a cooling system configured forsupplying a cooling fluid at a controlled cooling fluid flow rate tocool the heat-generating components; a temperature-based cooling controlcircuit including a temperature sensor configured for sensing atemperature of the heat-generating components and control logic forincreasing a cooling fluid flow rate in response to the temperatureexceeding a temperature threshold; and a power-based cooling controlcircuit configured for identifying and quantifying an increasing powerconsumption trend over a target time interval and, during a period thatthe temperature of the heat-generating components does not exceed thenext higher temperature threshold, increasing a cooling fluid flow rateto the electronic device in response to the magnitude of the increasingpower consumption trend exceeding a power threshold.
 16. Thefluid-cooled computer system of claim 15, further comprising: a serverincluding the heat-generating components.
 17. The fluid-cooled computersystem of claim 16, further comprising: a system management controllerincluded on a motherboard of the server, the system managementcontroller including the control logic for identifying and quantifyingan increasing power consumption trend over a target time interval andthe control logic for increasing a cooling fluid flow rate to theelectronic device in response to the magnitude of the increasing powerconsumption trend exceeding a power threshold during a period that thetemperature does not exceed the temperature threshold.
 18. Thefluid-cooled computer system of claim 15, further comprising: one ormore cooling fans, wherein the control logic for increasing a coolingfluid flow rate comprises control logic for increasing a fan speed ofthe one or more cooling fans.
 19. The fluid-cooled computer system ofclaim 15, further comprising: a liquid coolant heat exchanger in thermalcommunication with the one or more heat-generating components, whereinthe control logic for increasing a cooling fluid flow rate comprisescontrol logic for increasing a liquid coolant circulation rate throughthe heat exchanger.